A deep dive into the theory and applications of diffusion models, focusing on image generation and other tasks, with examples and PyTorch code.
The article discusses how machine learning is being used to calculate the macroscopic world that would emerge from string theory, a theory that posits the existence of tiny, invisible extra dimensions. These calculations have been difficult due to the enormous number of possibilities, but recent advances in artificial intelligence have made it possible to approximate the shapes of the Calabi-Yau manifolds, the objects that resemble loofahs and host quantum fields in string theory. The calculations have been able to reproduce the number of particles in the standard model, but not their specific masses or interactions. The long-term goal is to use these calculations to predict new physical phenomena beyond the standard model. The article also mentions that some physicists are skeptical of the usefulness of string theory and the role that machine learning will play in it.
An article discussing the use of Deep Q-Networks (DQNs) in reinforcement learning, which combines the principles of Q-Learning with function approximation capabilities of neural networks to address limitations of traditional Q-learning such as scalability issues and inability to handle continuous state and action spaces.
This paper introduces Cross-Layer Attention (CLA), an extension of Multi-Query Attention (MQA) and Grouped-Query Attention (GQA) for reducing the size of the key-value cache in transformer-based autoregressive large language models (LLMs). The authors demonstrate that CLA can reduce the cache size by another 2x while maintaining nearly the same accuracy as unmodified MQA, enabling inference with longer sequence lengths and larger batch sizes.
Additive Decision Trees are a variation of standard decision trees, constructed in a way that can often allow them to be more accurate, more interpretable, or both. This article explains the intuition behind Additive Decision Trees and how they can be constructed.
This article explains how to use Accumulated Local Effect Plots (ALEs) to understand the relationship between features and target in machine learning models, particularly when dealing with highly correlated features.
Google has launched Model Explorer, an open-source tool designed to help users navigate and understand complex neural networks. The tool aims to provide a hierarchical approach to AI model visualization, enabling smooth navigation even for massive models. Model Explorer has already proved valuable in the deployment of large models to resource-constrained platforms and is part of Google's broader ‘AI on the Edge’ initiative.
A surprising experiment to show that the devil is in the details
This article discusses causal inference, an emerging field in machine learning that goes beyond predicting what could happen to focus on understanding the cause-and-effect relationships in data. The author explains how to detect and fix errors in a directed acyclic graph (DAG) to make it a valid representation of the underlying data.
Fourier features in learning systems like neural networks due to the downstream invariance of the learner that becomes insensitive to certain transformations, e.g., planar translation or rotation.
Bookmarks on this page are managed by an admin user. 